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A  COMPARISON  OF  TRADITIONAL  AND  STEIN-RULE  ESTIMATORS 

UNDER  WEIGHTED  SQUARED  ERROR  LOSS 
George  G.  Judge  and  Mary  Ellen  Bock 

University  of  Illinois  at  Urbana-Champaign 

Using  a  weighted  squared  error  loss  measure  of  goodness  maximum  likelihood, 
pre  test  and  Stein  rule  estimators  are  analytically  compared  and  the  conditions 
necessary  for  one  estimator  to  be  superior  to  another  are  developed. 

1.  Introduction 

For  the  problem  of  estimating  the  mean  vector  of  a  multivariate  normal 

distribution  or  more  explicitly  for  our  purposes,  the  problem  of  estimating 

the  K  dimensional  parameter  vector  8_  for  the  orthonormal  regression  model 

(1.1)        v_  *  X 6  ♦  e  =  XS~V*S  +  e  *  Z6  ♦  e , 

where  y_  and  e_  are  (T  x  1)  normal  vectors  with  means  Z8_  and  £,  respectively, 

and  covariance  o2IT,  the  following  results  have  been  obtained  under  a  total 

squared  error  loss  measure  of  goodness,  with  risk  E [ (8  -  6) ! (£  -  8.)] J  i) 

James  and  Stein  [1961]  developed  a  Stein-rule  estimator,  0*  =  (1  -  c*/u)£, 

which,  for  K  >  3  and  0  <  c*  <  2(K-2)  (T-K)K*"1  [T-K+2]"1,  where  u  =  f/f/Ka2  =  F-K  T_K) 

is  the  likelihood  ratio  test  statistic,  dominated  the  conventional  maximum 

likelihood  estimator,  £  -  Z'yJ;  ii)  Baranchik  [1964]  and  Stein  [1966]  developed  a 

positive  part  variant  of  the  Stein-rule  estimator,  9  =  Ir  #   .  0-0  (1  -  c*/u)8, 

~    Lc  ,  °°)  — 

where  Ir  #  ^  (u)  is  a  zero-one  indicator  function,  that  dominated  the  James 

and  Stein  estimator,  6_*;  iii)  Strawderman  [1971]  developed  minimax  rules 

which  were  admissible  but  did  not  dominate  the  Stein  rule  positive  part 

estimator,  iv)  Sclove,  Morris,  and  Radhakrishnan  [1972]  showed  the  pre 

test  estimator,  6  =  Ir    n(u)6,  an  estimator  often  used  in  econometric  work, 
—    [c,  °°)   —  * 

is  uniformly  inferior  or  dominated  by  a  modified  version  of  the  positive 
part  Stein-rule  estimator,  6*  '  =  Ir    ,(u)Ir  rt  «a (u) (1  -  c*/u)£,  and  v)  Efron 
and  Morris  [1973],  using  empirical  Bayes  ideas,  showed  that  the  Stein  positive 
rule  estimator,  9_  ,  is  a  member  of  a  class  of  good  rules  that  have  Bayesian 


~2- 

properties. 

These  results  apply  to  the  |3  space  where  the  coefficient  variances 
and  weights  are  equal.   In  practice  nk-ny  situations  may  arise  where  the 
coefficient  variances  are  not  all  equal  or  we  wish  to  weight  certain  elements 
of  B_  differently  than  others.   Therefore,  the  purpose  of  this  paper  is  to 
summarize  the  conditions  necessary  for  the  James  and  Stein  estimator  [1961] 
and  its  variants  to  dominate  the  conventional  and  pre  test  estimators  for 
the  regression  statistical  model  under  a  weighted  squared  error  loss 
measure  of  goodness.   It  is  noted  that  under  situations  usually  found  in 
econometric  paractice,  where  non experimental  data  and  model  building  pro- 
cedures are  employed,  these  conditions  may  often  not  be  fulfilled. 

2.   Estimator  Comparisons 
To  compare  the  estimators  we  use  the  weighted  risk  function 

(2.1)     E[t|  -  evwcS  -  6)]  -  e[(|>  -  (ys^ws-^ce  -  03], 

where  W  is  a  (K  x  K)  symmetric  positive  definite  matrix  and  from  (1.1), 

u 

0  =  S  3.   If  as  a  special  case  the  weight  matrix  is  W  «  I.,  then  the  weight 

-W  -k    -1       -1 
matrix  in  the  0  space  is  S  S   =  5  "  ■  (X'X)   and  for  expository  purposes 

this  is  the  case  we  will  consider,  although  it  should  be  understood  that  the 

results  we  present  are  valid  for  any  positive  definite  W  matrix. 

In  the  comparing  the  estimators,  _9,  will  be  said  to  dominate,  iB, 
if 

(2.2)    E[(6  -  6)'S"^-/S"^(e  -  6)3  -  E[(£  -  6)  8S"\s"^(6  -  6)3  <  0, 
for  all  9_,  with  strict  inequality  holding  for  some  6L 

From  the  work  of  Bock,  Yancey  and  Judge  [1973]  and  Bock  [1974],  we 
may  write,  for  the  case  W  -  I™.,  the  risk  function  for  the  James  and  Stein 
estimator  0*,  as 


-3- 

(2.3)  E[(£*>  .ys'^S'^Ce*-  0J]  =  a2tr$~l   +  a2trS-1c*(T  -  K) 

F{(K  -f  2H)"l(K  -  2  +  2H)"1  [c*(T  -  K  +  2)  -  2(K  -  2) 
+  Ci'S'^/ePeCtrS"1})  2H(c*(T  -  K  ♦  2) 

-  2{3fe(trS"1}/i'S"1^  -  2})]i 
wjjere  H  is  a  Poisson  random  variable  with  parameter  £*8/o"2. 

In  order  for  the  risk  of  the  James  and  Stein  estimator,  9*,  to  be 
equal  to  or  less  than  the  risk  of  the  maximum  likelihood  estimator,  8_, 
over  the  entire  range  of  the  parameter  space,  0,  the  expression  between 
the  {}  brackets  must  be  zero  or  negative,  For  this  to  occur  and  thus 
E[(£  ""  £)'s"  C£  -  £)3  -  E[(£*  -  £)'S  (£*  -  £)]  l   0,  it  must  be  true  that 

K  -1 

(2.4)  I.  d./d,  =  trS  /d{.  >  2 

(2.5)  o  <  c*  <  2(T  -  KHtrS"1  d"1  -  2)/K(T  -  K  +  2), 

where  the  d.  are  the  roots  of  S"  ,  with  d.  being  the  largest.  For  the 

1  L 

general  weighted  case  these  roots  wou~d  be  associated  With  the  matrix 

s~\  s'1*. 

These  conditions  imply  that  under   the  weighted  squared  error  loss,  the 
superiority  of  the  James  and  Stein  estimator,  6*,  relative  to  the  conventional 
estimator,  §_,  depends  not  only  on  the  number  of  explanatory  variables  or 
hypotheses,  as  was  one  of  the  requirements  for  the  traditional  unweighted  loss 

in  the  _6  space,  but  also  on  whether  or  not  trS   or  the  sum  of  characteristic 

-i 

roots  divided  by  the  largest  characteristic  root  of  S   is  equal  to  or  larger  than  ! 

-1 

If  trS  /dL  <  2,  then  for  no  value  of  c  >  0  does  the  James-Stein  dominate 

the  least  squai-es  estimator. 


-4- 

Since  the  degree  of  col  linearity  of  the  columns  of  the  design  matrix 
X  is  related  to  the  magnitude  of  the  roots  of  S  ,  and  ill  conditioned  X'X 
matrix  may  affect  whether  or  not  the  risk  functions  cross  at  some  point  in  the 
parameter  space  and  thus  has  a  direct  impact  on  the  choice  of  estimator.  An 
analysis  of  the  design  matrix  for  several  auto  regressive  and  conventional  estimated 
economic  relationships  suggests  that  the  condition  trS^/d,  >  2   may  often 
not  be  fulfilled  in  econometric  work  and  therefore  the  James  and  Stein 
estimator  does  not  fulfill  the  minimax  criterion, 

These  same  conditions  or  requirements  (trS  Vd  >  2)  and  (2.S)  must 

Li 

also  hold  in  order  for  the  James  and  Stein  positive  part  estimator,  8_, 
and  the  extension  of  the  Sclove  modified  positive  part  Stein-rule  or  pre- 
liminary test  estimator,  6_  ,  to  dominate  the  least  squares,  §_,  and  con- 
ventional  preliminary  test,  £,  estimators,  respectively.  This  means  the 
appearance  of  three  or  Biore  regressors  and  a  suitably  small  c*  do  not  insure, 
as  they  did  in  the  unweighted  loss  (prediction  in  the  J3  space)  case,  that  the 
risk  of  these  various  extensions  of  the  Stein- rule  estimators,  will  be  less  over 
the  entire  parameter  space  than  conventional  and  pre  test  estimators ,  As  a 
consequence  the  condition  (2.4)  suggests  a  new  rule  for  determining  the  range  of 
d.  or  in  another  sense  the  coefficient  weights  or  degree  of  collinearity  per- 
missible among  the  columns  in  the  design  matrix  X  to  permit  these  extensions 
of  the  Stein-rule  estimators  to  dominate  the  conventional  sampling  theory  estimators. 

3.  Concluding  Remarks 
As  we  change  from  the  traditional  statistical  model  usually  analyzed  in 
the  literature  when  considering  Stein-rule  estimators  to  that  of  the  more 
general  regression  case  with  a  weighted  squared  error  loss  function,  the 
following  analytical  results  hold: 

i)     In  order  for  the  extended  Stein-rule  estimators  specified  in  this 
paper  to  dominate  conventional  estimators,  the  ratio  of  the  sunt  of 
the  characteristic  roots  of  the  (X'X)  matrix,  to  the  largest  root  of 


■ : 

this  matrix  must  be  equal  to  or  greater  than  2. 

ii)       If  this  ratio  is  not  greater  than  two  then  some  members  of  the 
family  of  potential  risk  functions  for  extensions  of  the 
Stein-rule     mators  problem,  cross  the  risk  function 

of  the  least  squares  est 

iii)       If  conditions  (2A   and  2.S)  are  fulfilled,  the  Sclove  modified 
Stein-rule  (pre  aator,  for  the  general  model,  is 

uniformly  superior  to  the  conventional  pre  test  estimator  over 
the  parameter  space.   Like  the  conventional  pre  test  estimator 

however,  its  risk  function  crosses  that  of  the  least  squares 
estimator  for  large  values  of  the  critical  value  c  or  small  values 
of  a;  the  Stein-ruie  only  yields  ,  ftinima  estimator  for  smaller 
values  of  c  or  larger  values  of  a  than  are  ordinarily  used  in  practice, 
iv)       The  incidence  of  col linearity  among  the  columns  of  the  X  matrix  to  the 
extent  that  the  Xs X  matrix  is  "borderline"  full  rank,  and  one  root  is 
small  relative  to  the  other  roots ,  means  that,  the  conventional  pre 
test  and  the   family  of .Stein-rule  estimators  are  superior  (smaller 
risk)  to  the  least  squares  estimator  only  over  a  small  interval  of 
the  parameter  space  and  are  inferior  (larger  risk)  over  a  large,  and 
in  some  esses,  infinite  range  of  the  parameter  space. 
The  ans^    :1  results  summarized  herein  suggest  that  unless  the  re- 
searcher has  great  confidence  that  his  linear  hypotheses  are  true,  under  the  risk 
measure  we  have  employed  when  col linearity  among  the  columns  of  the  design  matrix 
is  present,  there  is  much  to  lose  and  very  little  to  gain  by  broadening  the  class 
of  estimators  and  using  the  two  stage  pre  test  or  Stein-rule  procedures. 

Finally  we  note  the  rules  discussed  in  this  paper,  with  the  exception  of 
those  by  Strawderman  [1971]  do  not  satisfy  the  conditions  necessary  for  a  generalized 
Bayes  estimator  and  thus  are  not  admissible.  One  important  area  of  work  for  the 


-6- 

future  is  t0  evaluate  the  performance  of  these  under  alternative 

specifications  and  loss  In  regard  to  alternative  measures  of  per- 

formance it  shou       ited  that  BednareK-Kozek  [1973],  using    his  measure  of 

.1-   -  JU 

goodness  E[S*2(9  -  6  has,  my  making  use  of  the  work  of 

Stein  [1SS5]  demonstrated  ths       ?ast  s<    s  estimator  is  admissible. 
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